Apparatus, systems and methods for in vivo imaging

ABSTRACT

The disclosed apparatus, systems and methods relate to the use of optical nanoparticles in the illumination and imaging of tissues such as cancer tissues. Optical nanoparticles such as upconverting nanoparticles can be introduced into a patient and illuminated at a first time and wavelength and then imaged at a second time and wavelength to improve resolution and reduce imager size.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority to International PCT Application No.PCT/US20/49474, filed on Sep. 4, 2020, which claims priority to U.S.Provisional Application No. 62/895,757 filed Sep. 4, 2019, which ishereby incorporated by reference in its entirety under 35 U.S.C. §119(e).

GOVERNMENT SUPPORT

This invention was made with government support under grant no. R21EB027238 awarded by The National Institutes of Health and grant no.W81XWH-15-1-0531 awarded by The Defense Advanced Research ProjectsAgency. The government has certain rights in the invention.

TECHNICAL FIELD

The disclosed technology relates generally to surgical devices andtechniques, and in particular, to the devices, methods, and designprinciples for an intra-operative imaging system for visualizingmicroscopic disease. This has implications in the minimally disruptivetreatment of a variety of diseases.

BACKGROUND

The disclosure relates to apparatus, systems and methods for anintra-operative imaging system for visualizing microscopic disease.Undetected microscopic residual disease and lymph node involvement iscommon in curative cancer surgeries and significantly increases cancerrecurrence, driving the need for molecularly image guided cancersurgery. Successful treatment of early stage cancer depends on completeresection of all disease, both gross and microscopic, yet microscopicfoci of cancer cells, unable to be seen or felt, are all too often leftbehind, significantly increasing the chance of cancer returning acrossdisease sites, and in select cases, reduces survival. This often promptsadditional therapy consisting of broad areas of empiric re-excisionand/or radiation to reduce this increased risk, causing significanttoxicity (and cost), while potentially missing the residual diseasealtogether. This poses a particular challenge in breast cancer, whereclinicians and patients must weigh the morbidity associated withempirically resecting normal healthy tissue (mastectomy) against usingmore focal surgery (with radiation), but risk leaving tumor cells behindwith significant consequences. Microscopic residual disease (MRD)doubles the local recurrence (LR) rate, from 15% to 30% over 15 years,increasing breast cancer deaths, leading to a recommendation forre-excision in many cases. In breast cancer, this occurs with strikingfrequency: 20-40% of 140,000 women who undergo lumpectomy annually inthe US, are found to have MRD, leading to ˜37,000 reoperations. FIG. 2is a bar graph depicting the risk of recurrence with MRD in variouscancer types. Ultrasound, specimen radiography, MarginProbe and the likelack sensitivity for MRD. Blind, empiric removal of additional tissue isstill inadequate: a recent phase III randomized clinical trial comparinglumpectomy with or without empiric shave margins, still showed a 20%rate of MRD after shave excision (albeit reducing the 34% rate of MRDwith lumpectomy alone) attesting to the importance of not onlyidentifying MRD, but determining its exact location for re-resection.

Despite the advent of molecularly targeted imaging agents, image guidedsurgeries remain hindered by the intraoperative imagers themselves. Thisproject unlocks the power of inorganic optical nanoparticles tosignificantly advance in vivo intraoperative molecular imaging forcancer by developing an ultra-thin (<200 μm) molecular imaging skin,integrated on any surgical instrument, for real-time, single-cellvisualization of tumor cells intraoperatively.

Undetected lymph node involvement possesses an even greater challenge,as these lymph nodes lie in minimally dissected areas, often severalmillimeters below the tissue surface. Left untreated, they often recuras distant metastases, leading to death, as evidenced by several studiesshowing a survival benefit to empiric irradiation of lymph nodes(targeting MRD) in high risk patients. However, undetected MRD in lowerrisk patients continues to go untreated, and RT empirically delivered touninvolved nodes in even high-risk patients causes significant morbidity(pneumonitis, lymphedema>30-50% in patients with surgery+RT).

The lack of precision, individualized knowledge of tumor cell locationin vivo results in empirically administered large surgical orpost-operative radiotherapy fields, causing significant toxicity whilepotentially still leaving tumor cells untreated. Although fluorescentlylabeled targeted molecular agents accurately label single cancer cellsin vivo, the constraints on intraoperative imagers placed by organicfluorophores remain the limiting reagent: the small Stokes shift andabsorption cross-section require high-performance optical filters andlenses. The bulk and rigidity of these optics limits currentintraoperative instrumentation: to line of sight vision, missing themajority of the resection cavity and lymph node basins; to operationfrom outside the tumor bed, significantly decreasing sensitivity; andprecludes manipulation within the small surgical cavities inherent inmodern minimally invasive oncologic procedures.

Thus, there is a need in the art for improved devices, systems andmethods for the imaging of microscopic tumor foci, which cannot be seenor felt, are often left behind in the tumor bed and lymph nodes duringcancer surgery, increasing cancer recurrence and metastases,respectively.

BRIEF SUMMARY

Discussed herein are various devices, systems and methods relating to amicroscopic imaging system. Various of the disclosed examples utilizethe long UCNP time-constants on the order of 100-1000 μs, readilydetectable by modern high-speed silicon-based ICs, to implementtime-resolved imaging, enabling a filterless high-density imaging arrayto cover a large tissue area with high spatial resolution. Without theneed for optics, the silicon imager is thinned and can be scaled to awide range of dimensions. It can be coupled to a thin LED or laser diodeand placed directly on the tissue surface, achieving unprecedentedintraoperative mobility, increasing both sensitivity and spatialresolution through micron proximity to tumor cells, obtainingsingle-cell resolution. Various implementations include back-sideillumination, where in the long wavelength light goes through the backof the silicon-based imager because silicon is effectively transparentat wavelengths above 1100 nm and directly illuminates tissue through thechip itself.

In Example 1, an imaging system, comprising an imaging chip; and acomposition comprising at least one optical nanoparticle wherein theimaging chip is constructed and arranged to detect light emission fromthat nanoparticle.

In Example 2, the imaging system of claim 1, wherein the imaging chip isconfigured for time-resolved imaging.

In Example 3, the imaging system of claim 2, wherein the imaging systemfurther comprises an illumination device.

In Example 4, the imaging system of claim 1, wherein the imaging chip isfilterless.

In Example 5, the imaging system of claim 1, wherein the at least oneoptical nanoparticle has a luminescence lifetime greater than 1microsecond.

In Example 6, the imaging system of claim 1, wherein the at least oneoptical nanoparticle has a luminescence lifetime greater than 10microseconds.

In Example 7, the imaging system of claim 1, wherein the at least oneoptical nanoparticle comprises an upconverting nanoparticle (UCNP).

In Example 8, the imaging system of claim 1, wherein the at least oneoptical nanoparticle upconverts near-infrared light to higher energylight.

In Example 9, the imaging system of claim 1, wherein the imaging chip islensless.

In Example 10, the imaging system of claim 1, wherein the imaging chipis between about 25 microns and about 1 mm thick.

In Example 11, the imaging system of claim 1, wherein the imaging chipis between about 1 mm{circumflex over ( )}2 and about 40 cm{circumflexover ( )}2 wide.

In Example 12, the imaging system of claim 1, wherein the imaging chipis fitted to a medical device.

In Example 13, the imaging system of claim 12, wherein the medicaldevice is a scalpel, probe, drop in probe for laparoscopic or roboticsurgery, or glove.

In Example 14, the imaging system of claim 1, further comprising anillumination array.

In Example 15, the imaging system of claim 14, wherein the illuminationarray is an array of LED or laser diodes surrounding the chip.

In Example 16, the imaging system of claim 14, wherein the illuminationarray is an array of LEDs, laser diodes, or fiber optics configured forthrough illumination.

In Example 17, the imaging system of claim 14, wherein the throughillumination is patterned.

In Example 18, the imaging system of claim 1, wherein the opticalnanoparticle is conjugated to a molecule targeted toward a cell type ofinterest.

In Example 19, the imaging system of claim 18, wherein the molecule is aprotein, antibody, component of an antibody, or small molecule.

In Example 20, the imaging system of claim 19, wherein the cell type ofinterest is a cancer cell.

In Example 21, a method of imaging disease tissue, comprising:introducing a composition comprising at least one optical nanoparticleinto tissue; illuminating the at least one optical nanoparticle; andrecording luminescence of the at least one optical nanoparticle with animaging chip.

In Example 22, the method of claim 21, wherein the at least one opticalis illuminated at a first wavelength and emits light at a secondwavelength.

In Example 23, the method of claim 22, wherein the illuminationwavelength is longer than the emitted wavelength.

In Example 24, the method of claim 21, wherein the luminescence isrecoded after the at least one optical nanoparticle is illuminated.

In Example 25, the method of claim 21, wherein at least one opticalnanoparticle is an upconverting nanoparticle.

In Example 26, the method of claim 21, further comprising performingratiometric imaging to determine the depth of the at least one opticalnanoparticle.

In Example 27, the method of claim 21, wherein at least one opticalnanoparticle is conjugated to a molecule or protein that binds to atarget cell.

In Example 28, the method of claim 21, wherein the protein is anantibody or derivative.

While multiple embodiments are disclosed, still other embodiments of thedisclosure will become apparent to those skilled in the art from thefollowing detailed description, which shows and describes illustrativeembodiments of the disclosed apparatus, systems and methods. As will berealized, the disclosed apparatus, systems and methods are capable ofmodifications in various obvious aspects, all without departing from thespirit and scope of the disclosure. Accordingly, the drawings anddetailed description are to be regarded as illustrative in nature andnot restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A-1H depict several views of the components and implementations ofthe system.

FIG. 2 is a bar graph depicting the risk of recurrence with MRD.

FIG. 3 depicts a workflow wherein nanoparticles with unique opticalproperties (including long luminescent lifetimes, >10 microseconds, andup conversion, an example of which are upconverting nanoparticles orUCNPs) are injected intratumorally or systemically, labeling tumor cellsin vivo. These nanoparticles can be conjugated to targeted molecules,such as small molecules and antibodies (referred to as immunotargettedUCNPs). Tools with the molecular imager embedded on their surface thenresect (Example 1) or are inserted into tissue (Example 2), such as in alumpectomy/prostatectomy or biopsy, respectively. The real-time (<0.01s) images are projected onto a monitor and automated image recognitionsoftware identifies and quantifies the cells seen.

FIG. 4 is an overview of integrated imaging of single cancer cells withan ultrathin imaging chip embedded in surgical instruments.

FIG. 5 is an overview of the imaging of upconverting nanoparticlesaccording to one implementation. (A) shows the absorption, transfer, andemission in Yb3+/Er3+-based aUCNPs following either NIR-I (980 nm) orNIR-II (1550 nm) excitation. (B) is a cartoon of imaging platformdirectly integrated onto surfaces, such surgical glove. (C) is a diagramof the time-resolved image acquisition scheme. Texc, pulse excitationtime; Tint, emission signal integration time; Tdelay, time gating delay.(D) shows angle-selective gratings used to achieve lensless imageacquisition and block obliquely incident background light.

FIG. 6 depicts a view of an imager chip, according to oneimplementation. where each component of a conventional fluorescencemicroscope is transformed into microfabricated planar layer, oreliminated altogether. The prototype integrated circuit imaging chipalone is then capable of molecular imaging of cells labeled with aUCNPs.

FIG. 7 is a photodiode pixel level photodiode and circuitry convertphotons to electronic signals.

FIG. 8 depicts a setup INSITE configuration for imaging of aUCNPdispersions. Laser beam width is 300 μm

FIG. 9 : Time-resolved Imaging (TRI) timing diagram, and measured 60/40NaYbErF4 aUCNP time constant of using 25 μm thinned imager.(AU=arbitrary units)

FIG. 10 shows UCNP emission measured with custom time-resolved imagingarray. Emission decay (a) instantaneous and (b) integrated, showingbackground dark current. Emission vs illumination (c) power and (d)duration (peak 5 ms).

FIG. 11 depicts UCNP Decay versus Composition. Emission decays of UCNPs(Tint=1 ms, Texc=5 ms) as measured by pixel output at (A) 8 W/cm2 of 980nm excitation, or (B) 60 W/cm2 of 1550 nm excitation. Tint isintegration time; Texc is duration of excitation light pulse.

FIG. 12 shows integrated visible emission as a function of UCNPcomposition at either 980 and 1550 nm excitation, both with 8 W/cm2excitation power density. Hexane blank is without UCNPs.

FIG. 13 (a) is aUCNP-TRI array setup, (b) imaging a slit of aUCNPs (1000images), (c) Single image with increasing integration times (1, 3, 5 ms)showing SNR and SBR of 4 with single 5 ms image. (d-f) Spatialresolution of imager with ASG, and (f) anticipated improvement withproposed imager.

FIG. 14 shows chip imaging of UCNPs. (a) aUCNP-TRI array setup, (b)imaging a 500 um microwell of aUCNPs in a PDMS mold showing sharpfeatures (d-f) Spatial resolution of fluorescence contact imager withASG, and (f) anticipated improvement in resolution with proposed theimager.

FIG. 15 (A) is a photograph of CMOS contact imager and PDMS micro-wellholding UCNPs, fabricated for this study. (B) Time-resolved image of theUCNP-coated surface of the micro-well. Normalized emission intensityshown as in grayscale legend. Section (I) is background, (II) is outerPDMS with UNCPs diffused into PDMS, and (III) is microwell with UCNPsample. (C) Cross-section emission profile for the three regions in (B).

FIG. 16 shows in vivo imaging. (a) Margin imaging We have demonstratedboth systemic and intratumoral injections of fluorescently-tagged(IR700DX) Trastuzumab in a HER2+ breast cancer model (left mouse). In(b) a positive margin is identified in real time (<0.05 s) with thefluorescence chip-based imager (confirmed with IVIS imager), despite novisible tumor in the resection bed. (c) Chip imaging of aUCNPs in an invivo prostate cancer model. LNCaP cells are implanted bilaterally and asingle tumor is injected with aUCNPs and visualized with the customizedIVIS live animal imager, outfitted with a 980 nm laser, showing aUCNPsin the tumor. Non-injected (−) and injected (+) tumors are excised andimaged (d,e), with the 980 nm laser focused on each one sequentially. Nobackground autofluorescence is seen in the (−) tumor and aUCNPs arevisible in the (+) tumor. (f) The (+) tumor is diced in to 6 pieces(underlined); aUCNP containing-tumor (square) is placed on the bareimager chip (g, no lenses, no filters), and illuminated with 1550 nmlight. The chip identifies aUCNPs within the tissue (h). Live mouseimages of intratumorally-injected NaEr0.8Yb0.2F4 aUCNPs with 8 W/cm2 980nm excitation. (I) Images of aUCNPs-injected into mouse prostate tumor(left) and non-injected side (right). Emission intensity as in coloredlegend. (J) Measured emission spectrum of the injected and non-injectedsides, showing tumor-specific Er3+ emission bands at 545 and 655 nm

FIG. 17 is on imager. Photograph of the intratumorally-injected tumorsample placed directly on the contact imager.

FIG. 18 depicts tumor imaging. (A) is a microscope image of excisedtumor injected with 25 μL of a 250 nM aqueous dispersion of UCNPs,excited with 1 W/cm2 980 nm, showing distinct localization of UCNPs. (B)shows images of NIR-II laser scanning of tumor, from top to bottom inincrements of 300 μm (numbered 1-8). At each position, an image of thetumor sample is acquired with INSITE using a 5 millisecond-pulsed 60W/cm2 1550-nm laser for illumination.

FIG. 19 shows upconversion through-illumination of silicon. 980 and 1550nm is transmitted through a thin layer of silicon from a back-sideattached LED array. Unconverted photons are absorbed by photodiode,producing an image.

FIG. 20 shows device integration, according to one implementation. (a)shows 100 μm-thin microdiodes affixed to the back of the CMOS imagerchip (c), which is thinned to 25 μm (b).

FIG. 21 depicts model chip architecture/in-pixel architecture withsubstrate carrier artifacts addressed in next gen design to allow 10-40W/cm2 of through-illumination. (b) time-resolving timing diagram, and(c) measured time constant of thinned imagers, with 25 μm imager showingonly a 75 μs substrate carrier time constant, much less than aUCNPs.

FIG. 22 shows an imager thinned to 25 microns and shown in comparison toa human hair.

FIG. 23 shows several views of integrated photopnics. (a) Focalillumination at the pixel level, similar in principle to confocalimaging, allows deconvolution of the image for increased resolution. (b)A similar principle can be applied to two sensors facing each other. (c)Example of Global Foundries silicon nanophotnics, with micron-scalephotonics integrated with on-chip circuitry.

DETAILED DESCRIPTION

The various systems and devices disclosed herein relate to devices foruse in medical procedures. More specifically, various embodiments relateto imaging devices, systems and methods for visualizing microscopicresidual disease (MRD) and microscopic lymph node involvement (mLNI) inan intraoperative environment. Certain exemplary implementations relateto imaging systems, devices, and methods for visualizing microscopicbreast and prostate cancer. It is understood that the variousembodiments of the system and related methods disclosed herein can beincorporated into or used with any other known medical devices, systemsand methods, including those disclosed in co-pending U.S. applicationSer. No. 15/687,205, filed Aug. 25, 2017 and entitled “Apparatus,Systems and Methods for Intraoperative Imaging,” which is acontinuation-in-part of U.S. application Ser. No. 15/074,614, filed Mar.18, 2016 entitled “Methods, Systems, And Devices For Imaging MicroscopicTumors,” which was filed under 35 U.S.C. § 371 and claims the benefit ofInternational PCT Patent Application No. PCT/US14/56788, filed on Sep.22, 2014, and U.S. Provisional Application 61/880,750, filed Sep. 20,2013, and entitled “Methods, Systems, And Devices For ImagingMicroscopic Tumors,” and U.S. Provisional Application No. 62/379,416filed Aug. 25, 2016 and entitled “Apparatus, Systems and Methods forIntraoperative Imaging,” each of which is hereby incorporated byreference in its entirety. While the system is amenable to variousmodifications and alternative forms, specific embodiments have beenshown by way of example in the drawings and are described in detailbelow. The intention, however, is not to limit the invention to theparticular embodiments described. On the contrary, the invention isintended to cover all modifications, equivalents, and alternativesfalling within the scope of the invention as defined by the appendedclaims.

Prior art imagining approaches have limitations. Despite theintroduction of targeted molecular agents for cancer surgery guidance,the requisite bulky and rigid optics of prior art devices necessary forimaging conventional fluorophores preclude maneuvering within in small,minimally invasive tumor cavities, and imaging the entire surface areawith high sensitivity (<200 cells) as well as accessing lymph nodebasins. Standard postoperative pathologic evaluation of excised tumorspecimens can result in sampling error, challenges with co-registrationback to the tumor cavity, and can leave lymph node involvementunchecked.

Recognizing the importance of integrating molecular imaging with cancersurgery and radiation across cancer types, such as in the treatment ofbreast and prostate cancer, the disclosed system 10 for molecularimaging that removes the restrictions of current bulky imagers, enablingseamless molecular imaging in virtually any tissue contacted by asurgical instrument such as a scalpel or glove.

In various implementations, the disclosed imaging system 10 andassociated devices and methods are used to image tissue through the useof optical nanoparticles such as upconverting nanoparticles. In variousimplementations, the system 10 utilizes different times and/orwavelengths to image. That is, in certain implementations, the systemilluminates the tissue at a first wavelength and images or absorbsemitted light at a second wavelength. Further, in certainimplementations, the system 10 is used to illuminate the tissue at afirst time and image it at a second time, such as after the illuminationhas ceased. This is made possible through the use of opticalnanoparticles that emit or luminesce absorbed light at a secondwavelength (higher or lower than the first wavelength) and/or opticalnanoparticles that emit or luminesce after illumination has ceased, incertain implementations for 1 or more microseconds longer. In furtherimplementations, the system 10 can be configured to operate in apatterned or time-resolved matter, as described herein. Other opticalnanoparticles can be used that have the properties of upconversion andlong luminescent lifetimes (greater than one microsecond). These can beof a variety of compositions, including modifications of the outer shellcoating, polymer wrapping and surface functionalization (to allow forattachment to targeted molecules such as proteins, small molecules andantibodies, or derivatives thereof).

Turning to the drawings in greater detail, FIGS. 1A-H depict an overviewand various aspects of the system 10 and associated methods and devices,according to certain implementations. As shown in FIGS. 1A-1B, incertain implementations, the system 10 comprises an imager 14 mounted ona surgical device 12, certain non-limiting examples being a glove 12 orscalpel 12, though it is readily appreciated that many other alternativedevices are of course possible, such as mounting the imager 14 on astandalone probe or biopsy needle, or integrating the imager 14 intosurgical robotic tools and the like. In these implementations, theimager 14 is constructed and arranged to detect signals 2 emitted fromtissue 4 by upconverting nanoparticles (UCNPs) 6. Variousimplementations of the system 10 for the direct integration of an imager14 with surgical tools 12 are outlined here.

According to various implementations, the imager 14 is a high-speedintegrated circuit (IC) imaging array configured to be used withbiocompatible alloyed UCNPs 2-3 orders of magnitude brighter thanconventional UCNPs to image disease tissue 4. Such use of UCNPs 6 areable to make use of both an extended luminescence window—a long opticaldelay—and the characteristic of the UNCPs flourescence to higherenergies to avoid the need for the lenses and/or filters of prior artimaging devices, see, for example.

As such, the present system 10 and associated devices and methodsrelieve the cumbersome optical requirements imposed by organicfluorophores in favor of an optics-free microfabricated, scalable,time-resolved contact imaging (TRI) array imager 14. The system 10thereby enables molecularly-guided resection and targeted irradiation ofadvanced cancers far more precisely than current empiric-basedstrategies, eliminating the 30% of cancer surgeries that areunsuccessful due to the inability to visualize microscopic disease, andtransforming postoperative radiation therapy (PORT) planning withprecision localization of tumor cells.

By taking advantage of imaging from within the body, microns tomillimeters away from the subject tissue, the various implementations ofthe system 10 achieve single-cell intraoperative imaging throughout thesurgical bed by dispensing with limitations of organic fluorophores infavor of UCNPs, which, as shown for example in FIGS. 1A-E, can beconjugated to antibodies (UCNP-Ab) targeted to cancer-specific markers.

Conceptually, using only a single preoperative injection of UCNPs-Ab,tumor cells are precisely labeled in vivo. Made possible byhighly-efficient UCNPs 6, which allow for 2-photon imaging at fluencescompatible with hand-held in vivo applications, optics are eliminated byshifting the requirement for color imaging to the time domain, wellsuited for high-speed electronics using the unique UCNP properties ofnear-infrared absorbance and upconversion to image tissue 4 labeledUCNPs 6 such that the emitted light 2 is at a higher energy than theabsorbed light 1.

As shown in FIG. 1H, the near-infrared (about 980 nm) absorbance allowssimple through-illumination of silicon (transparent at thesewavelengths), deep into tissue, while the upconversion—a uniquelyefficient ability to absorb photons in the near-infrared range (NIR) andemit light 2 at higher energies—transforms the emitted signal into aneasily imaged, visible photon devoid of background and autofluorescence.

Unfortunately, no clinical imaging methods are capable of detectingmicroscopic disease intraoperatively; conventional imagers such as CT,MRI and PET are limited to a resolution of ˜1 cm³ or 10⁹ cells, ordersof magnitude above the ˜200-cell threshold needed to image foci of MRD.The limited ability to identify microscopic tumor foci stems from boththe imager's distance from the tumor cells, and a lack of definitivemolecular identification, instead relying on inferred characteristicssuch as size and contrast enhancement. Strategies relying on cellmorphology such as touch-prep or frozen section require an on-sitepathologist and are complicated by the fatty nature of breast tissue,limiting their utilization.

Intraoperative fluorescent imaging. The self-limiting depth penetrationof light reduces background while conveying useful information about thetissue surface and several millimeters below, ideal for tumor margin andlymph node imaging. Using targeted molecular agents, multiple animalstudies demonstrate tumor labeling in vivo with fluorescently-labeledsystemically injected antibody. Recent clinical trials employ thisstrategy to label tumor cells and guide resection intraoperatively,spanning ovarian, pancreatic, esophageal (NCT02129933) and breast(NCT01508572) cancers. Imaging agents targeting more general markers oftumor, such as matrix metalloproteinases (MMP), are in clinical trials,expanding applicability of this platform to a wide range of cancers.However, conventional organic fluorophores drive the need for bulky,rigid optics, precluding effective intraoperative imaging.

With an absorption cross section of approximately 10⁻¹⁶ cm², quantumyield of ˜10%, and Stokes shift of only ˜50 nm, fluorescence imagingrequires dual optical paths, a high performance optical filter, andfocusing optics. Current intraoperative fluorescent imagers, are largelybased on a similar principle of placing a large microscope above thetumor bed and are inadequate for two key reasons: (1) they arerestricted to line of sight only, as rigid optics are required forfluorescent imaging (optical paths for excitation and emission light,focusing objectives), missing the majority of the tumor cavity surface;and (2) They image far from the tumor bed, restricted by the size ofoptics to outside the tumor cavity, reducing resolution and sensitivityas light diverges over the distance squared. Scaling current fluorescentmicroscopes to mm-scale encounters fundamental limitations asfabricating optics at the micro-scale introduces significant aberrationdue to imperfections, and imagers are still several centimeters inlength. Light-sheet microscopy requires external optics, precluding usein the tumor bed. Fiber optic approaches lack the maneuverabilitynecessary for small lumpectomy cavities, facing a fundamental tradeoffbetween flexibility and fiber diameter, significantly limiting the areaimaged for a flexible fiber.

Given that imaging devices themselves remain the limiting factor intranslating the high-contrast molecular imaging of the pathology labinto the operating room to identify MRD anywhere in the surgical field,the various implementations of the system 10 dispense with opticsaltogether, embedding a photosensitive imaging skin into any surgicaldevice. By placing the imager skin directly onto the tissuesurface—increasing sensitivity via micron proximity while being smallenough to manipulate within the cavity—the system 10 allows the user tothoroughly image the entire surface with high sensitivity.

To image single-cells directly within tissue, the variousimplementations of the system 10 represent a new imaging technology,that directly integrates into an ultra-thin, planar form-factor, that isembedded directly on surgical instrumentation, transforming the toolitself into a single-cell molecular imager (As shown in FIG. 1 and FIGS.3-4 ) to guide precision cancer surgeries and biopsies, or any otherinvasive procedure. That is, FIGS. 3A-3B depict system 10 workflowaccording to two examples wherein immunotargeted aUCNPs 6 are injectedintratumorally or systemically, labeling tumor cells in vivo.Instruments 12 with the molecular imager 14 embedded on their surfacethen resect (Example 1) or are inserted into tissue (Example 2), such asin a lumpectomy/prostatectomy or biopsy, respectively. The real-time(<0.01 s) images are projected onto a monitor 100 and automated imagerecognition software identifies and quantifies the cells seen, as wouldbe readily understood.

FIG. 4A-4D depict a further views of integrated imaging of single cancercells with an ultrathin imaging chip 14 embedded in surgical instruments12, according to certain implementations of the system 10 showing thetissue and in vivo imaging of the labeling.

FIGS. 5A-5D are an overview of the system 10 according to certainimplementations where the system 10 is upconverting and imagingnanoparticle. FIG. 5A shows the absorption, transfer, and emission inYb3+/Er3+-based aUCNPs following either NIR-I (980 nm) or NIR-II (1550nm) excitation. FIG. 5B depicts an imaging platform directly integratedonto surfaces, such surgical glove. FIG. 5C is a diagram of thetime-resolved image acquisition configuration of the system 10, whereTexc is pulse excitation time, Tint is emission signal integration timeand Tdelay is time gating delay. FIG. 5C shows that angle-selectivegratings can be used to achieve lensless image acquisition and blockobliquely incident background light in certain implementations.

FIG. 6 depicts an imager chip 14 according to certain implementations ofthe system 10. It is appreciated that each component of a conventionalfluorescence microscope (such as illumination, camera and filters) istransformed into microfabricated planar layer on the IC 14, oreliminated altogether. That is, the imaging chip 14 of FIG. 6 isconfigured for time-resolved imaging (TRI) and comprises a backsideillumination source 16 such as a μLED array. Further, in lieu offocusing optics, the integrated circuit imaging chip 14 alone is thencapable of molecular imaging of cells labeled with aUCNPs. FIG. 7 is apixel-level schematic showing a photodiode and circuitry convert photonsto electronic signals

The various implementations of the system 10 introduce a fullyintegrated molecular imaging system, consisting of an optical label (forexample, UCNPs) and a custom optics-free, high-sensitivity time-resolvedintegrated circuit. Enabled by the unique optical properties of UCNPsand similar particles, the various implementations of the system 10eliminate or miniaturize each component of a conventional fluorescencemicroscope, resulting in an ultra-thin planar, imaging surface (See,e.g., FIGS. 1 and 4A-5D). The surface can have thin flexible wires toconnect it to an external computer and monitor for data processing andvisualization.

Our key innovation is the synergistic integration of a novel molecular,<1 mm thin (and can be reduced to ≤25 microns thin (see below at FIG. 22), imaging skin made from integrated circuit technology and UCNPs,embedded on the surface of any surgical tool, transforming the toolitself into a molecular imager (shown for example in FIG. 1 ).

In certain aspects, the system 10 comprises an imager 14 having a highlysensitive imaging array in integrated circuit technology, turning thecamera into a single chip. The imaging chip itself can be thinned to <25μm and scaled to cm² dimensions, realizing a molecular imaging skin.

In certain aspects, the system 10 utilizes the long luminescent decaytimes of specific optical labels (such as UCNPs 6) to shift thecumbersome requirements of color (fluorescence) imaging into the timedomain, synergistic with modern high-speed integrated circuits,eliminating the need for optical filters.

Optical filters are eliminated using time-resolved imaging: alternatingpulses of illumination light and imaging only the decaying fluorescentsignal emission, when the illumination light is off (FIG. 7B). Thisrequires a long-luminescent lifetime optical particle.

The various implementations of the system 10 eliminate lenses usingmicron proximity to tissue and on-pixel angle selective gratings fordeblurring.

Sensitivity is enhanced by proximity, gathering light before it divides,and low-noise circuit design.

Spatial resolution is achieved without lenses through micron-proximityto tissue, integration of on-chip angle selective gratings (7 μm tallstructures integrated directly on photodiodes) and a high pixel density.

The various implementations of the system 10 eliminate conventionalillumination sources and optical light guides through direct integrationof light sources (for example LEDs or laser diodes) to the backside ofthe imaging chip, shining NIR light directly through silicon(transparent at these wavelengths) and deep into tissue.

Through-illumination solves challenges in illuminating tissue beneaththe opaque contact imager: By fixing light sources (such as lightemitting diodes or laser diodes, or fiber optics) that emit infraredlight at a wavelength longer than about 1000 nm, directly to the back ofthe silicon-based imager, which is transparent at these wavelengths,tissue is directly illuminated through the chip itself. This eliminatesboth relatively inflexible fiber optics and inefficient side-coupledillumination, which requires an optical particle with IR absolutiongreater than about 1000 nm. However, because Stokes shifted light willbe even a longer wavelength and therefore pass through silicon-basedphotodiodes, we require visible to near IR light to be emitted from thetissue being imaged, which requires an optical particle capable of upconversion (bound to the tissue). Up conversion also eliminatesautofluorescence since no naturally occurring tissues have thisproperty.

Conversion of light to an electronic image at the sensing point obviatesfiber optics without loss of sensitivity. As such, inflexible opticsfibers are eliminated by converting the optical signal to an electricalone at the tissue surface using integrated circuit-based contact imager.

Integration of silicon nanophotonics for a monolithically integratedsensor. Integration of on chip illumination using silicon nanophotonicsallows light to be routed on chip. This allows light to be pattered atthe pixel level. The resulting pattered illumination can increase thespatial resolution beyond that of the imager alone.

Depth imaging using ratio metric imaging. By illuminating nanoparticlesthat can absorb at two different wavelengths, and each wavelength hasdifferent absorbances in tissue, the relative intensities of the twoimages can provide information about tumor depth. In one application forUCNPs, this works because illumination light at 980 nm will pass to adepth “d” with a different, intensity (and loss) than illumination lightat 1550 nm. Using this principle we can derive the dimension informationabout the distribution of nanoparticles (and the cells they label oridentify) below the tissue surface. Calibration of ratiometric imagingcan be done on tissue phantoms prior to use for imaging. Ratiometricimaging can be used with any of the illumination strategies outlinedherein.

In implementations where LEDs or laser diodes are used, two or morewavelengths can be used and alternated. In implementations where fiberoptics are used a single fiber optic can be used and two different lightsources can be alternated sequentially. Similarly, a fiber optic bundlecan be used and different wavelengths of light can be channeled throughdifferent individual fibers within the bundle and illuminatedsequentially. Two or more wavelengths can be used, such as wavelengthsdiffering by about 50 nm or more. Here, examples of 980 nm and 1550 nmwere used, but any wavelengths can be used that are absorbed by ananoparticle and cause optical emission by that nanoparticle.

Optical labels. Conventional fluorophores have extremely fast timeconstants (nano second)—making time—resolved imaging challenging for anarray-based sensor, required for spatial resolution and rapid imagingover a large surface area, where thousands of pixels must functionsimultaneously. Few fluorophores absorb in the infrared, and two photonprocesses necessary for upconversion are highly inefficient, requiringpower levels incompatible with in vivo use.

It is appreciated that any organic or inorganic optical label that can:upconvert with a lifetime of greater than about 1 ms lifetime iscompatible with this system. That is, any label that can upconvert byemitting at a higher energy (shorter wavelength) after absorbing lowerenergy (longer wavelength) photons for more than about 1 ms iscontemplated by the system.

One such optical label are upconverting nanoparticles. Examples includeupconverting nanoparticles (UCNPs) and UCNPs with shells, which allowlonger radiative lifetimes. References include and commercial productssuch as Creative Diagnostics (DNL-H011 DiagNano™ PEG-NH2 UpconvertingNanoparticles, 545 nm/660 nm, DNL-H003 DiagNano™ PEG-COOH UpconvertingNanoparticles, 545 nm/660 nm). Any other particle or optical label thatfulfills these requirements can also be used.

UCNPs simultaneously address both these challenges, enabling afirst-in-class imager with their uniquely long luminescent lifetimes(>100 μs), IR absorption, and up-conversion, allowing development of anarray-based time-resolved imaging platform using only CMOS(complementary metal oxide semiconductor)—enabling a first-in-classscalable, ultrathin molecular imaging array. Substrates for the imagingchips and photosensors include silicon, germanium, gallium phosphide(GaP), indium gallium arsenide (InGaAs), indium arsenide antimonide,lead sulfide, diamond-based photodetectors and mercury cadmium telluridephotodiodes and other similar substrates understood and appreciated inthe art. Existing methodologies allow for protein conjugation toinorganic nanoparticles, including UCNPs. Nanoparticles with core-shellshave longer lifetimes, and techniques in coating including new materialsin shells around the core, can increase optical lifetimes.

The disclosed Examples demonstrate the feasibility of time-resolvedimaging of UCNPs with the imaging array. The UCNP decay time constant issufficiently long for array-based time-resolved IC imaging, (2) spatialfeature recognition drawing from the previous demonstration influorescent contact imagers using angle selective gratings, and (3) liveanimal imaging of UCNPs linking in vivo biodistribution experiments withimager findings.

A custom ultra-thin high-sensitivity imager. Starting with a custom2,880-pixel prototype image sensor, with a scalable architecture madefrom integrated circuit (IC) technology, the various implementations ofthe system 10 design the sensor to be an ultra-sensitive detector ofphotons, such as is shown in FIG. 6 . Each 44 μm pixel consists of aphotodiode, converting an incident photon into an electronic signal, asshown in FIG. 7 , amplified by in-pixel circuitry.

As shown in the incorporated references, ultra-rapid (<0.1 s)fluorescence imaging of tissue and cellular foci (down to 20 cells)using a custom, lensless integrated circuit imaging platform ispossible. the demonstrated low noise design approaches the shot-noiselimit, representing the fundamental limits of optical detection. Thiseffectively replaces a conventional camera with a single “chip”—readilythinned to just 25 μm microns, as is demonstrated in FIG. 17B. However,fluorescence imaging requires both optical filters and fiber optics orwaveguides to guide light into tissue, hindering direct integration intosurgical instruments. Furthermore, background tissue autofluorescenceand incomplete rejection of illumination light through the filtersignificantly limit sensitivity at the single-cell level. The variousimplementations of the system 10 overcome these limitations through theintegration of optical nanoparticles, such as UCNPs, with dualproperties of upconversion and long decay lifetimes.

Upconversion enables imaging without background. Since silicon isrelatively transparent to 980 nm and 1550 nm—the two absorption peaks ofthe UCNPs— it cannot detect a conventionally Stokes shifted photon—whichwould be at a lower energy (longer wavelength) and even more challengingto detect. However, the various implementations of the system 10 takeadvantage of upconversion and easily capture the emitted photon ofimmunotargeted UCNPs bound to tumor cells in the visible spectrum,readily absorbed by silicon. Upconversion has another distinctadvantage—no background tissue autofluorescence. Without background, theimager noise (and dark current) is designed to be below the intrinsicshot noise of the tumor signal, achieving single-cell detection.

Imaging within the optical window of tissue. It was proposed to use 980nm illumination for in vivo imaging, as it falls within the opticalwindow (low absorbance region) of tissue. However, given that 1550 nmhas greater absorbance in tissue, it has the advantage of imaging closerto the tissue surface (reducing background from non-specific bindingwithin the tissue), useful for sensitive margin imaging. Since UCNPs canabsorb at both wavelengths, in the Examples described herein,implementations of the system 10 were used to sequentially image at 1550nm and 980 nm, thereby imaging both at the surface and deeper intotissue, respectively.

Deriving depth imaging using combinations of 1550 nm and 980 nmillumination. The various implementations of the system 10 describedherein are able to determine the relative depth of labeled cells byexploiting the dual optical absorption windows of UCNPs, which absorblight at both 1550 and 980 nm. Tissue, labeled cells (or any substancecontaining UCNPs) is illumination sequential with 1550 nm light and animage is taken, and then 980 nm light, and another image is taken. Therelative intensities of these images, with the different depthpenetration, absorption and scatter properties of the excitation lighttaken into account (as well as any shifts in emission spectrum with 1550nm excitation versus 980 nm excitation), can be used to derive thelocation in tissue of the UCNPs.

Time resolved imaging (TRI) and the elimination of optical filters. Thevarious implementations eliminate optical filters, required to removethe illumination light—which is many orders of magnitude stronger thanthe 2 or 3 photon, upconverted emission light. Conventional fluorescenceimagers use specialized optical interference filters to rejectexcitation light by a factor of 10⁶ or more—which in turn require opticsfor precision alignment. Here, the various implementations of the system10 eliminate the need for optical filters altogether by taking advantageof the long luminescent lifetimes (˜100s of microseconds) to enabletime-resolved imaging (TRI) (FIG. 5 Overview).

Long decay lifetimes enable time-resolved imaging in modern CMOStechnologies, and alleviate the need for optical filters entirely. Whiletime resolved imaging has been demonstrated with organic and proteinfluorophores, their nanosecond radiative lifetimes make large, densearray based imaging impossible, as arrayed CMOS sensors cannot readilydetect on timescales shorter than tens of microseconds. Although singlephoton avalanche diodes that require specialized fabrication processeshave been demonstrated to operate at these timescales, they arechallenging to fabricate in a high density, massively parallelarray-based approach to enable large spatial coverage, high fill factor,and adequate spatial resolution necessary for efficient chip-basedimaging. Consequently, a chip-based imager using time resolved imagingrequires optical probes with microsecond lifetimes, such as upconvertingnanoparticles. The long (>1 microsecond) decay lifetimes open the doorto time-resolved imaging in an array-based CMOS imager.

Time resolved imaging (also called time-gated imaging). The variousimplementations of the system 10 illuminate with a pulse of light,transiently exciting the optical nanoparticles (e.g. UCNPs), and imagewithin microseconds after the excitation light is turned off— completelyeliminating background light, such as in FIG. 5 . This process can berepeated, and images can be averaged together to improve image quality(signal to noise ratio or SNR). and sequential images can be averaged toimprove image quality. The pulse of light can be any length of time thatresults in illumination and subsequent optical emission from theparticle. This can be as short as 1 nanosecond, to as long as severalseconds. In practicality, we select the pulse duration to be theduration that results in the maximal optical emission, which for theUCNPs tested here is approximately 1 millisecond. This can be customizedfor each nanoparticle. Pulse durations beyond that are not necessary.Shorter pulses (that achieve adequate optical emission from thenanoparticles) are advantageous because they reduce the total imagingtime, and allow for either more rapid imaging, or more serial images tobe taken and averaged, improving the image quality (SNR). After theillumination light pulse is off, the image is taken. The illuminationlight is off for the duration of the image. There is a small delay(Tdelay) between the illumination light being turned off and the imagetaken, and this can be adjusted. This delay is to allow any transientsin the imager chip as a result of the illumination to decay. The imageis taken with a set integration time. The image can be stored on chip,transmitted off chip, subjected to signal processing, digitized or anyother standard process for an image.

The present imager according to certain implementations is a massivelyparalleled pixel array, captures the decaying, upconverted, emissionwith high spatial resolution. The relatively long UCNP lifetime allowsintegration times on the order of 100-1000 microseconds, which areachievable in modern CMOS processes.

The chip-based time-resolved imaging (also referred to as time-gatedimaging) method of the current system 10 takes advantage of uniquelylong emission lifetimes for select optical labels with one example beingUCNPs. The long decay lifetimes alleviates the need for high performancefrequency-selective (color) filters by separating the emission andexcitation signals in the time domain rather than in frequency domain, astrategy that can be implemented in modern high-speed integrated circuitdesign. In a chip-based imager, the various implementations of thesystem 10 implement this by briefly pulsing the excitation light (forexample, T_(exc)=5 ms duration although any excitation duration can beused sufficient to impart energy to the optical label) while the imagingpixels are not integrating. After the excitation light is turned off,the pixels are turned on, and integrate the emission signal from theUCNPs for 1 ms, as shown in FIG. 5 . Since there is no backgroundexcitation light at this time, the need for an optical filter iseliminated. The illumination and signal acquisition scheme (FIG. 5C),where the excitation light source is pulsed for T_(exc), andsubsequently turned off, after which the emission signal from the UCNPsis acquired and integrated by the imager. Any interference and unwantedsignal caused by the excitation light can be rejected by delaying(Tdelay) the integration window start point.

In order to determine whether an ASIC can image optical labels withoutthe use of conventional focusing lenses and optical filters, the variousimplementations of the system 10 are designed and fabricated an imagingarray capable of time resolved imaging, as shown in FIG. 5 . Absentoptics, the imaging chip can be easily thinned to just 25 microns andplaced directly on tissue, increasing sensitivity throughproximity—capturing light from optical labels before it diverges. Thekey to this platform is the transformation of molecular imaging from thecolor (frequency) domain to the time domain enabled by UCNPs. Thevarious implementations of the system 10 accomplish this usingsynergistic design of modern integrated-circuits and upconvertingnanoparticles. A key advantage of using CMOS-based imaging platform isthe ability to integrate in-pixel electronics enabling signal processingdirectly on-chip, eliminating the need for optical lenses. This imageraddresses both elimination of lenses and filters simultaneously, makingit possible to obtain optical images with a much smaller form-factor.

Optical characterization. UCNPs emission and lifetimes werecharacterized as functions of illumination intensity, illumination pulseduration, each with either 980 or 1550 nm excitation. Vials ofhydrophobic UNCPs dispersions in hexane (400 μL of the 0.68 μM) wereplaced above the imager array and excited with time-resolved collimatedlasers. The beam was positioned 2 mm above the surface of the imager.

INSITE samples were excited with a 980-nm wavelength-stabilized,single-mode, fiber-coupled laser diode (Qphotonics QFBGLD-980-500)followed by an adjustable collimator (Thorlabs ZC618FC-B) set to a beamdiameter of 1.27 mm; or a 1550-nm single-mode, fiber-coupled laser diode(Qphotonics QFLD-1550-1505) collimated by an aspheric collimator(Thorlabs CFS2-1550-APC) with a beam diameter of approximately 0.3 mm.Both lasers were driven by a temperature-controlled mount driver (ArroyoInstruments 6310 ComboSource).

Radiative lifetimes (τ) were modeled as a single exponential decay andwere calculated by extracting decay profiles with a fixed movingintegrating window (T_(int)). Assuming the dark current intensity(i_(d)) is constant over time, the various implementations of the system10 derive the integrated pixel value I_(A)(t) from the current densityi(t):

${{i(t)} = \left. {{i_{0}e^{({- \frac{t}{\tau}})}} + i_{d}}\rightarrow{{I_{A}(t)} \equiv {\int_{t}^{t + T_{int}}{{i(u)}{du}}}} \right.}{{I_{A}(t)} = {{{\underset{I_{A_{0}}}{\underset{︸}{i_{0}{\tau\left\lbrack {1 - e^{({- \frac{T_{int}}{\tau}})}} \right\rbrack}}}e^{({- \frac{t}{\tau}})}} + \underset{I_{d}}{\underset{︸}{i_{d}T_{int}}}} = {{I_{A_{0}}e^{({- \frac{t}{\tau}})}} + I_{d}}}}$

where τ and I_(D) are the emission decay lifetime and dark current levelin the pixel, respectively. Dark current level was subtracted fromwaveforms.

UCNP Sensitivity and Lifetime Measurements. To evaluate the sensitivityof the imaging platform for UCNPs the various implementations of thesystem 10 utilize and approach like that depicted in FIG. 8 . thepreliminary data demonstrates imaging UCNPs within the safe andallowable ASNI power limits—which reflect higher allowances for pulsedlight (over continuous illumination). The various implementations of thesystem 10 have successfully imaged 0.68 μM UCNPs, over a volume of 3 μl(corresponding to 5×10⁵ cells at 2×10⁷ UCNPs/cell, equivalent to 1×10⁷cells at 10⁶ UCNPs/cell which is the expected number of aUNCPs perlabelled cell) at 980 nm and 1550 nm. With millisecond pulses of 6 and65 W/cm², respectively, at a relatively large 2 mm distance from thesensor surface, the various implementations of the system 10 detectUCNPs with an SNR of 26 dB and 22 dB at 980 and 1550 nm (as shown inFIGS. 9-10 and in Table 1).

FIG. 8 shows the measured decay profile of the nanoparticles using theimager, thinned to just 25 μm, demonstrating an exponentially decayingphotocurrent for the UCNP emission with a long lifetime of t_(UCNP)=1.1ms, establishing feasibility of this approach with the prototype.Similarly, we have performed time-resolved imaging of several UCNPconstructs with varying proportions of Yb and Er (at both 1550 nm and980 nm (shown in FIG. 11E). 60/40 NaYbErF4 has the brightest emissionand longest lifetimes at both 980 and 1550 nm, as shown in FIG. 11 ,leading us to select this construct for further development.

As shown in FIG. 10 , UCNP imaging shows the measured decay profile ofthe nanoparticles using the imager (1 ms integration time),demonstrating an exponentially decaying photocurrent for the UCNPemission with a long lifetime of t_(UCNP)=1.1 ms. No backgroundautofluorescence is visible due to the fact that their emissiondissipates within pico to nanoseconds. The relative signal to backgroundratio (SBR) is measured by integrating the signal (serially increasingintegration time from 1 to 30 ms), and optimal SBR of 3 is achievedafter a 5 ms integration time, as shown in FIG. 10 . As background issolely due to dark current, this will be reduced in subsequent designs.

As further demonstration, the various implementations of the system 10image a series of core/shell UCNPs with varying Yb³⁺ and Er³⁺ content tomeasure emission using either 980 nm or 1550 nm photoexcitation, asshown in FIG. 11 . UCNP cores (8 nm) were synthesized with 20/80, 40/60,or 80/20 Yb³⁺/Er³⁺ ratios and overgrown with inert 4-nm shells. Withthese nanocrystals in the experimental setup, shown in FIG. 8 , theinitial emission intensity, A·, luminescence lifetime, τ, and totalintegrated emission count, A·τ, were measured at a given power density(8 W/cm² at either 980 and 1550 nm). Measured decays for varying UCNPcompositions range from 600 μs to 1.3 ms at either 980 nm or 1550 nmexcitation. These values are longer than measured or calculated, butconsistent with UCNP power-dependence given the lower excitation powersused here.

To demonstrate proof of concept, the present study characterized imagingensemble UCNPs with the imager. This example demonstrates visualizationof UCNP using the chip-based imager by imaging a series of core/shellUCNPs with varying Yb³⁺ and Er³⁺ content to measure emission usingeither 980 nm or 1550 nm photoexcitation. UCNP cores (8 nm) weresynthesized with 20/80, 40/60, or 80/20 Yb³⁺/Er³⁺ ratios and overgrownwith inert 4-nm shells. UCNPs in hexane (with hexane alone serving asthe negative control) were excited (pulsed illumination) with 980 nm and1550 nm, source and imaged with a custom 2,880 pixel array, sized at 55μm each, implemented in 180 nm technology, externally controlled by anFPGA to accurately manipulate timing.

The effects of excitation (T_(exc)) pulse duration on emission signalintensity were assessed. To extract the excitation duration dependency,the UCNPs were excited for increasing durations of time (Texc) and theemission intensity was measured. This duration represents how long thenanoparticles are illuminated with the excitation light source beforethe start of the time-resolved imaging sequence.

With these nanocrystals in the experimental setup depicted in FIG. 8 ,in various implementations of the system 10 the initial emissionintensity, A·, luminescence lifetime, τ, and total integrated emissioncount, A·τ, were measured at a given power density (8 W/cm² at either980 and 1550 nm). Measured decays for varying UCNP compositions rangefrom 600 μs to 1.3 ms at either 980 nm or 1550 nm excitation. Thesevalues are longer than measured or calculated, but consistent with UCNPpower-dependence given the lower excitation powers used here.

While most Yb³⁺/Er³⁺ upconversion is nominally a 2-photon process withfollowing 980-nm excitation of the Yb³⁺²F_(5/2) manifold, 1550 nmexcitation of the Er³⁺⁴I_(13/2) manifold leads to upconversion via anominal 3-photon process, for example as shown in FIG. 5A. For all UCNPcompositions, NIR-I excitation produces a stronger signal than withNIR-II, consistent with significantly higher efficiency of 2-photonversus 3-photon upconversion processes.

Optimizing UCNP Emission Intensity for Use With Imager. Besides theconcentration of the nanoparticles, illumination power and durationaffect the emission signal's intensity. The flux of emission photons is,to first order, proportional to the influx of excitation photons andnonlinearities are seen only at very large excitation powers, where theparticles start becoming saturated. It important to note that thevarious implementations of the system 10 are visualizing ensembles ofUCNPs at fluences of <1 W/cm², compatible with in vivo use. Increasingthe duration of illumination of the UCNPs, during which the imager isoff, increases UCNP emission intensity, eventually saturating andreaches its final value near t=5 ms.

Spatial Resolution of Images. Eliminating Optics. Finally, the variousimplementations of the system 10 eliminate the need for focusing optics,which are challenging to miniaturize while maintaining performance.Optics suffer from fundamental limitations such the significantaberration and imperfections present when fabricating optics at themicroscale (and other issues such as ghost images) and miniaturizedfluorescence imagers are still several centimeters in length. By placingthe imaging chip directly against the tissue itself, an approach called“contact imaging”, the micron-proximity of the imager to tumor cellscaptures light before it diverges—preserving spatial resolution andincreasing sensitivity. The various implementations of the system 10combine high pixel density and in-pixel integration of micro-fabricatedangle selective gratings (ASG) for image deblurring (7 μm tallstructures integrated directly on photodiodes which the variousimplementations of the system 10 have demonstrated), to enable singlecell resolution, as shown in FIG. 10 .

This imager chip is thinned to just 25 μm and back-side coupled to a 100μm-thin LED for a completely integrated molecular imaging “skin” whichis embedded within surgical instrumentation. Connection via a thinflexible wire to an external computer and monitor allows for imagingprocessing, cell recognition and visualization, such as is shown abovein FIG. 6 .

The various implementations of the system 10 have demonstratedquantification of cell detection using an integrated circuit-basedfluorescence imager, shown in FIGS. 14A-14D. Building on that work, thevarious implementations of the system 10 have developed a CMOStime-resolved imaging array. Images are obtained with the 2,880 pixelarray, implemented in 180 nm technology, externally controlled by anFPGA to accurately manipulate image timing. The closer the imagersurface is to the target cells, the higher the sensitivity and spatialresolution. Previously, the various implementations of the system 10have demonstrated a spatial resolution of ˜250 μm imaging cells usingthe custom fluorescence contact imager with angle-selective gratingshowever, resolution for fluorescence is limited by the 500 μm opticalwaveguide and filter required to insert light between the opaque imagerand tissue surface.

FIG. 14C is simulated data that shows a microscope image, and thecorresponding image from the custom sensor, FIG. 14D, of 3D cellcultures. In use, time-resolved imaging and through-illuminationeliminates the need for both filters and waveguides, and simulating thesame image with separation of just 100 μm (FIG. 14E), the spatialresolution is improved >5×, reducing background and blur and enablingsingle-cell identification.

Spatial Resolution and Determination of Minimum Detectable Signal. Thepresent examples also demonstrate time-resolved imaging of spatialfeatures with UCNPs in a 500 μm microfabricated well to simulate a focusof labeled cells (FIGS. 14-15 ). Recognition of small tumor foci amidsta strong and varying background of nonspecific binding andautofluorescence requires accurate background subtraction. Variations inantibody distribution and binding (both within, and between, patients)cannot be predicted a priori, prohibiting the use of a global thresholdfor MRD and current efforts are limited to centimeter-scale tumor foci.Hence, certain implementations require “local” background measurements,near the tumor cluster, for accurate subtraction and edge detectiondriving the need for spatial resolution below the millimeter-scaleexcision achievable by surgery. Alternate implementations do not requirebackground measurements.

To determine the imaging quality achievable with a 25-micron thinmicroscope, the implementation of the system 10 shown in FIG. 15A, a1550 nm excitation source was used to excite the UCNP-coatedmicrostructure and the image was acquired using time-resolved imaging.The custom-fabricated PDMS micro-well and the CMOS imager demonstratethat INSITE is able to resolve the spatial features of the micro-wellwith nearly single-pixel sharpness (55 μm), translating into a spatialresolution performance sufficient for detecting microscopic residualdisease. A cross-section of the acquired signals (shown in FIG.15B-15C), shows three different regions of the image. Aside from themicro-well and the background, the intermediate zone represents the PDMSsurrounding the micro-well. Due to the porosity of the PDMS, smallamounts of UCNPs diffuse into the surrounding area, generating a smallsignal in this area. For a concentration of 10¹² UCNPs per mm², themeasured average signal-to-background ratio is 6.5. The background isdominated by the dark pixel current, which can ultimately be subtractedout.

INSITE achieves this spatial resolution without the use of conventionallenses through both proximity to the tissue sample and directintegration of on-chip microfabricated collimators, and is limited onlyby the pixel size. INSITE uses angle-selective gratings (ASG) to improvespatial resolution with chip-based imaging. ASG are arrays ofmicrocollimators fabricated directly on each pixel, as described indetail in the incorporated references and in, for example Papageorgiou,E. P., Boser, B., & Anwar, M. (2019). Chip-Scale Angle-Selective Imagerfor In Vivo Microscopic Cancer Detection. IEEE Transactions onBiomedical Circuits and Systems, using only the inherent metalinterconnect layers common to all CMOS process—obviating the need forany postprocessing and not adding any thickness to the imager itself.The versatility of CMOS fabrication technology has led to the on-chipintegration of a variety of optical components such aswavelength-selective optical filters that could be tuned to becompatible with quantum dot applications, or stacked diffractiongratings for lensless 3D imaging to reject angled incoming light anddecrease blur in the image. Other lensless imaging platforms have alsobeen reported in that leverage computational techniques. As demonstratedhere, the elimination of optical filters and focusing optics enablesplacement of the custom designed INSITE imaging chip directly againstthe sample itself, capturing light before it diverges, achieving bothspatial resolution and increased sensitivity without optics.

Strategies to increase image quality include increasing excitation laserpower, and increasing the number of images acquired for averaging. Sincevarious implementations of the system 10 are pulsing light, thosevarious implementations of the system 10 use the time-averaged power,allowing the instantaneous illumination to reach 80 W/cm² (for short,<10 s, durations). Further approaches are of course possible.

Illuminating at Safe Optical Intensities. To image a single cell withthe prototype sensor, certain implementations of the system 10 determinethe minimum illumination intensity needed. The number of receptors (e.g.HER2, PSMA) per cancer cell varies from 10⁵/cell to 10⁷/cell, thus to beselective these implementations of the system 10 aim at detection of 10⁶UCNPs within one pixel. By using short (millisecond) pulses of light,higher instantaneous illumination power can be used, while maintaininglow total power delivered. The ANSI (skin exposure) limit for a short 2ms pulse of 980 nm is 420 W/cm² (total power 0.85 J/cm²) and 500 W/cm²(1 J/cm²) for 1550 nm.

Image Recognition. To identify tissue labeled by UCNPs with this imager,the various implementations of the system 10 employ of cell cluster sizeusing an automated cluster recognition algorithm as described inPapageorgiou, E. P., Zhang, H., Giverts, S., Park, C., Boser, B. E., &Anwar, M. (2018). Real-time cancer detection with an integrated lenslessfluorescence contact imager. Biomedical Optics Express, 9(8), 3607-3623,or other appropriate image recognition strategy.

Tissue Imaging. As shown in FIGS. 16A-16H, in vivo labeled tissuesamples have previously been imaged with an fluorescence-basedintegrated circuit imager 47-50 and live animal imaging of UCNPs inmammary glands (recapitulating a breast cancer model. The variousimplementations of the system 10 also have established proof-of-conceptfor tissue imaging of UCNPs with the custom imager in a breast andprostate cancer model (FIGS. 16A-16B): UCNPs are injected into a mousebearing a PSMA-expressing LNCaP tumor, imaged (FIG. 16C) and thenexcised. The excised tumor (and negative control uninjected tumor) areimaged with both the customized live animal imager (FIG. 16E) and imaged(FIG. 16G) at 1550 nm excitation light. The chip readily visualizesUCNPs (FIG. 16H) in the tumor tissue.

To determine the applicability of the imaging system in tissue imaging,a prostate tumor was injected with aqueous 250 nM polymer-encapsulated26-nm NaEr_(0.8)Yb_(0.2)F₄ aUNCPs. UCNP-injected mice were imaged acustom-modified IVIS imager using NIR-I illumination, showingcolocalization of the tumor and UCNPs. Images of a tumor on thecontralateral side of the mouse without UCNP injection shows nomeasurable visible emission (background dark current only). To ensureUCNPs were being imaged, the spectrum of the acquired emission signalwas measured (FIG. 16I-16J), displaying the characteristic emissionspectrum of the nanoparticles, with the two major visible emission bandsof the UCNPs at 545 nm and 655 nm. Using the UCNP emission band at650-670 nm, the signal-to-background ratio is 15.

To determine if the UCNPs injected into the tumor can be visualized withINSITE alone, the injected tumor was excised and imaged with the INSITEchip imager. FIG. 17 shows a photograph of the excised tumor sample onthe 25-micron imaging chip. For reference, the image of the excisedtumor on a microscope is shown in FIG. 18A, and the image acquired withINSITE reveals a distinct area of UCNPs in the excised tissue is shownin FIG. 18B, with a signal to background ratio of 9, closely matchingthe performance of the IVIS imager, which incorporates high performanceoptical filters and a cooled CCD camera. Results from these experimentsdemonstrate that the ultra-thin time-resolved CMOS imager, customdesigned to image engineered UCNPs, is capable of imaging within tissue,with no background autofluorescence and with little to no additionalinterference from the excitation light. More extensive work intoxicology and biodistribution with immunotargeted UCNPs will be neededto assess the suitability for these nanoparticles for tumor targeting inmice or human tissue.

Through-Illumination enables UCNP imaging in a single monolithic imagingchip. To fully realize a molecular imaging skin with a contact imager,the tissue surface must be illuminated—however the path to directillumination is blocked by the imager itself. Strategies such as sideillumination introduce additional bulk with integrated LEDs, LaserDiodes or VCSELs; however, silicon itself is transparent in the infraredwith a bandgap at 1.1 eV or 1100 nm, as shown in FIG. 19 . Illuminationin the IR range allows direct penetration through the imager, with afurther advantage of deep tissue penetration, but requires up-conversionto ensure that emitted photons interact with the photosensor. Here, thevarious implementations of the system 10 are configured for lightillumination integration with LEDs, Laser Diodes or the like and imagersconfigured to allow for (a) mitigation of substrate carriers and (b)pixel saturation, generated by high-intensity through-illumination. Itwill be appreciated that light penetration below the band-gap isexponentially related to silicon thickness, driving the desire for anultra-thin substrate.

Integrating illumination enables a fully integrated system requiringonly ultrathin, flexible wires to communicate power and data—enablingrealization of a molecular imaging skin. the preliminary dataestablishes the feasibility of through illumination, and introducesseveral technical advances needed to move to illumination powers neededfor in vivo imaging.

Through Illumination Using LED or Laser Diode Backside Integration. Oneoption for illumination is to integrate a thin (1 mm or less) LED. LaserDiode, VCSEL, or other light emitting device with NIR (>900 nm, forexample either 980 nm or 1550 nm for UCNPs) emission onto the back ofthe imager to enables a fully integrated solution. The light emittedfrom the backside of the imager passes through the imager andilluminates the sample (such as tissue) on the other side (imaging side,with photodiodes). In this method, light passes through the imager, andis thus called through illumination. The wavelength of light must besuch that the imager material (in this example, silicon) is transparentto it. In the case of silicon, transparency increases with longerwavelength light (such as light above about 900 nm, with improvedtransparency above about 1100 nm).

Optical power loss must be accounted (for example, up to 50×) due to (1)reflection due to the high index of refraction of silicon and (2)absorption within the silicon, which is exponentially related to siliconthickness. The illumination light can be patterned to improve spatialresolution whereby fine patterns of light are used to illumination thesurface, often at a spatial resolution that is finer than the imageritself. The illumination patterns can be sequentially altered and theimages taken after each illumination patterned can be mathematicallycombined to create a higher resolution image. This principle is known aspatterned or structured illumination. Here, the patterns can be used asa method for time-resolved imaging, where by each patterned is pulsedone or more images, and the image taken after each pulse.

Reflection on entry to the silicon requires index-matching layers (gels,example ThorLabs G608N3) to optically couple into the chip, reducingoptical loss from reflection by 2×. A commercially available LED wasaffixed to the thinned chip. One such example includes SMBB970D-1100from Epitex, a 1 mm thick package with 2 W output at 970 nm. This diecan be removed from the package and further thinned to achieveultra-thin monolithic integration (<200 μm).

Substrate carriers are generated by light entering the chip substratefrom backside, or through illumination. These carriers drift into thephotodiode causing background, indistinguishable from signal, andtherefore imaging must occur after these carriers have recommended. Therecombination lifetime can be written t_(life)=W²/(2*Dn) where W is thewafer thickness and Dn is the Diffusion coefficient for electrons(approximately 36 cm²/s in silicon). The strong dependence on waferthickness stems from carriers recombining at a high rate at the siliconsurfaces. Using the first-generation imager, we measured the effect ofsubstrate carriers on pixel output as a substrate thickness,demonstrating decreasing t_(life) as a function of chip thickness(t_(life)=75 μs at 25 μm wafer thickness, as shown in FIG. 8C).Recognizing that the LED now represents the dominant thickness of thedevice, ultra-thin LEDs can be integrated.

FIG. 20 shows device integration, according to one implementation. (a)shows 100 μm-thin microdiodes affixed to the back of the CMOS imagerchip (c), which is thinned to 25 μm (b). The resulting molecular imaging“skin” can be affixed to surgical instrumentation, with only thin wiressupplying power, data and control signals to the chip. As an example, ascalpel will with imagers on both sides will be developed.

Chip Design. Substrate carriers are generated by light entering the chipfrom backside or high-power through illumination. These carriers driftinto the photodiode causing background, indistinguishable from signal,and therefore imaging must occur after these carriers have recommended.The recombination lifetime can be written t_(life)=W²/(2*Dn) where W isthe wafer thickness and Dn is the Diffusion coefficient for electrons(approximately 36 cm²/s in silicon). The strong dependence on waferthickness stems from carriers recombining at a high rate at the siliconsurfaces. Using the first-generation imager, we measured the effect ofsubstrate carriers on pixel output as a substrate thickness,demonstrating decreasing t_(life) as a function of chip thickness(t_(life)=75 μs at 25 μm wafer thickness, as shown in FIG. 21C.

Further Imaging Implementations. While the first-generation imager workswith through-illumination, a new pixel design is needed for increasedthrough-illumination intensities, as substrate carrier concentrationsgenerated with >0.1 W/cm² create substrate currents that overwhelm thein-pixel amplifier through diffusion into both the photodiode andMOS-capacitors, used for sample and hold and correlated double sampling.The various implementations of the system 10 imager decouple typicalelements from the substrate such as eliminating the MOS-capacitors infavor of metal-insulator-metal capacitors and provide a transientlyincreased bias current during through illumination to prevent amplifiersaturation from the large photodiode current during illumination.

Certain implementations of the chip are constructed and arranged toinitially tolerate 40 W/cm² settling within less than 1 ms to allow foroptimal imaging. However standard design techniques can accommodatesignificantly more power. Challenges with amplifier settling, should itsaturate, can be mitigated with a longer delay time (t_(ΦR)), at thecost of lower integrated UCNP signal. An optimized photosensitive CMOSprocess (X-Fab) with P-I-N diodes can improve responsivity 10×,decreasing integration time.

In further implementations of the imager, the imager is constructed andarranged to decouple circuit and photodiode elements from the substratesuch as eliminating the MOS-capacitors in favor of metal-insulator-metalcapacitors and provide a transiently increased bias current duringthrough illumination to prevent amplifier saturation from the largephotodiode current during illumination.

Certain implementations of the chip make use of PNP photodiodes, wherebythe Nwell-Psub acts as a shield from the substrate.

Certain implementations feature switches that couple the photodiodedirectly to the power supply during illumination to avoid drawingsignificant current through the amplifier and saturating it. Challengeswith amplifier settling, should it saturate, can be mitigated with alonger delay time (t_(ΦR)), at the cost of lower integrated UCNP signal.

Certain implementations replace sample and hold capacitors withsubstrate independent capacitors (like MOSCaps) to avoid coupling intothe substrate. If MOS-Caps must be used, use PMOS devices to shield thetransistors from the substrate.

Various implementations feature an optimized photosensitive CMOS process(X-Fab) with P-I-N diodes that can improve responsivity by 10× or more,thereby decreasing integration time.

These designs provide the possibility to allow time-resolved NIR-II andNIR-I excitation of molecularly labelled cancer cells particles using athrough-illumination method. This illumination scheme would obviate theuse of any light-guiding structures such as lenses, crystals orwaveguides conventionally used to properly direct to and focus theexcitation light on the target molecules. For example, this could beused for an imager during surgery that was a drop-in probe placedagainst tissue of interest labeled with an optical nanoparticle and usedto image that tissue for nanoparticles.

As such, the various CMOS imagers have been designed to be robustagainst external and direct NIR excitation by a combination of designmodifications. Due to the absorption and sensitivity of Si-basedphotodiodes to NIR-excitation, unwanted carriers will be generated inthe substrate and will introduce interference on the pixel value. Thespecific type of photodiode used in the design is fabricated using a p+implant in an n-well sitting in the p-type substrate. The imager arraytakes advantage of the “p+/n-well” photodiode which is surrounded byanother internal photodiode, “n-well/p-sub”, which is used to drain themajority of the charges generated due to the NIR excitation duringillumination.

It is appreciated that as such, the NIR light is capable of travelingmuch further, and will therefore generate the majority of the carriersin the p-sub and n-well region and can therefore be drained by alow-resistance path (direct bypass to supply) in the n-well.

After the illumination, the resulting visible emission light from thespecimen will be captured and absorbed by the Si-based photodiode, andbeing visible and quickly absorbed, the majority of the emission lightwill be absorbed in the p+/n-well photodiode and can be shielded fromother interfering carriers in the substrate and n-well.

An additional draining mechanism for generated interfering carriers hasbeen implemented internally in the pixel front-end circuitry, byproviding a temporary low-resistance path from the p+ implant to groundto quickly drain and provide recombination current to the small amountof carriers generated in the p+ region as a result ofthrough-illumination.

The internal storage capacitors in the pixel have also been implementedusing metal-on-metal (MOM) capacitors, a much more robust alternative totheir MOS-type counterparts which are highly sensitive to substrate andcarrier disruptions.

Leakage of the internal reset switch in the pixel is also very criticalto the linearity and integrity of the output signal and image. To thisend, the input switch has been designed to generate a constant andsignal-independent leakage, that can be subsequently subtracted andproperly cancelled out. Deep integration of the circuits and/ornanophotonics enables a single, monolithic imaging sensor. Leveraging anunprecedented level of integration of electronics and photonics, madepossible by the recent introduction of an advanced GlobalFoundries 45 nmSOI process semiconductor manufacturing process, various implementationsof the system 10 directly embed illumination at the pixel level toachieve a higher spatial resolution using techniques similar to confocalmicroscopy. Image resolution is a product of both the pixel density andthe spatial illumination patterns. This allows both improved spatialresolution and the ability to retain all illumination and imaging in asingle CMOS process, allowing the entire chip to be just 25 micronsthin.

Typically, illumination is uniform, and resolution is a purely afunction of imaging optics and pixel density. Conversely, confocalmicroscopy uses a low-resolution imager, but precisely defines theillumination pattern, which then defines the spatial resolution. Here,various implementations of the system 10 combine these techniques,matching the pixel size with illumination patterns. With the currentpixel size of 55 μm, the goal is illumination precision on this order,and pixel size can be readily reduced below this in modern CMOSprocesses. Demonstrating the tight integration of electronics andnanofabricated optics, a team recently developed the world's firstmicro-processor with integrated photonic I/O, featuring 70+ milliontransistors and thousands of photonic components on the same die.Building on this and illustrating applicably to biosensors the group hasdeveloped a chip-based molecular sensor using nanophotonic resonators,as shown in FIG. 23C.

To fully integrate illumination with imaging, with no added thickness,various implementations of the system 10 route photonic waveguides andgrating couplers within the pixel array, allowing ultra-fine controlover illumination patterns at the pixel level. Sequential illumination(or in patterns) and imaging, as shown in FIG. 23A, enables a higherresolution image along the plane (X-Y) of the sensor to be resolvedwithin tissue. While confocal imaging has been used to image in thesub-cellular regime for table-top microscopes, the goal is to enablecellular level (˜10-100 micron) resolution in tissue, sufficient tomonitor changes in immune cell concentration and distribution indicativeof immune activation. These images can be “deconvolved” to gain improvedspatial resolution.

A similar strategy can use structured illumination where patterns oflight are displayed on the object (e.g. tissue containing UCNPs), andspatial features that are too fine to be visualized with the nativeresolution of the imager can be derived from the interference patternsof the illumination pattern and object. This illumination pattern isthen rotated, shifted, or otherwise changed, to allow multiple images toconvey the full information of the object, including features that areotherwise too fine (small) to be detected by the imager using uniformillumination.

To improve resolution in the (Z) plane perpendicular to the sensor,opposing sensors are placed facing one-another with a spacing ˜5 mm andimage simultaneously. At 700-800 nm illumination, common to clinicallyused near infrared dyes, light penetrates ˜5 mm (with 0.5% of lightremaining), allowing the same cells to be imaged by both sensors,enabling reconstruction of a 3-D image, as shown in FIG. 23 .

The various implementations of the system 10 can also use a techniquesimilar to light sheet microscopy, whereby the illumination light isfocused on a specific plane of the tissue and rastered through it.Similarly, certain implementations of the system 10 can use a techniquelike confocal microscopy, whereby light is focused at a single point andthe image is reconstructed.

Various implementations feature enhancements for single cell imaging.Various implementations feature one or more of the following features orimprovements: (1) safely increasing the optical illumination power bytaking advantage of increased power allowance for pulsed light versuscontinuous illumination, (2) decreasing the distance from thephotodiodes on the imager surface from millimeters to microns, (3)increasing intrinsic photodiode sensitivity by 10× through the use of animproved CMOS process, (4) lowering imager noise 2×, and (5) decreasingpixel size for improved spatial resolution.

Table 1 depicts improvements in SNR and minimum number of UCNPsdetectable with next generation imager. Image time: 2 ms illumination, 2ms readout. Single cell imaging with just 25,000 UCNPs/cell is possibleat 980. It is appreciated that American National Standards Institutelimits for a 2 millisecond pulse.

To image single-cells directly within tissue, the system 10 represents anew imaging technology, that directly integrates into an ultra-thin,planar form-factor, that is embedded directly on surgicalinstrumentation, transforming the tool itself into a single-cellmolecular imager (FIG. 9 ) to guide precision cancer surgeries andbiopsies.

The various implementations of the system 10 therefore relate to a fullyintegrated molecular imaging system through synergistic integration ofimmunotargeted UCNPs and a custom optics-free, high-sensitivitytime-resolved integrated circuit. Enabled by the unique opticalproperties of UCNPs, the various implementations of the system 10eliminate or miniaturize each component of a conventional fluorescencemicroscope, resulting in an ultra-thin planar, imaging surface withsingle-cell sensitivity requiring only thin, flexible wires to connectit to an external computer and monitor for data processing andvisualization.

With increased illumination intensity and closer placement of the tissuesample to the imager surface, the various implementations of the system10 expect detection of a single HER2+ breast cancer cell (10⁶UCNP/cell). Imaging though both blood and tissue is addressed in theprevious work, showing only a 6 dB loss (and therefore visible with 980nm illumination) through an opaque 250 um thick layer of blood. Inalternate implementations, SNR of cells labeled with immunotargetedUCNPs can also be performed in vitro with 3D cell culture models.

To achieve single-cell sensitivity for a wide range of cells, includingthose with low surface tumor marker expression, the variousimplementations of the system 10 improve sensitivity and spatialresolution. The various implementations of the system 10 only requirespatial resolution such that any non-specific background integratedwithin a single pixel does not mask the signal from the tumor cell.Thus, the various implementations of the system 10 do not requireresolution of individual cells. Increased sensitivity allows for lowerillumination power, and detection of single cells using a lowerconcentration of UCNPs delivered to the patient. The variousimplementations of the system 10 improve sensitivity through using aCMOS technology process with a 10× higher photon sensitivity. Thevarious implementations of the system 10 improve spatial resolution bycombining the in-pixel angle selective gratings for image deblurring andreduction of pixel size to 20 μm×20 μm, approaching the size of singlecell (10-15 μm). Pixel sizes can however be made to be less than 1×1um{circumflex over ( )}2.

Reduction in pixel size 4× to 20 μm×20 μm will improve spatialresolution 4×. Placement of the cell<100 μm from the surface increasesresolution by an additional 5× (total 20×). Optical sensitivity isincreased 10× (0.5 NW) by fabricating the sensor in an IC technologyprocess optimized for photodetection (XFAB Foundry) (SNR+10 dB) usingPIN photodiodes, as opposed to the relatively low-efficiency photodiodes(0.05 A/W) in the current 0.18 μm TSMC mixed signal process. Since thefunctional portion of the imager is contained within the superficial 10μm, chips are then thinned to 25 μm (a procedure readily done withintegrated circuits, as demonstrated in the above.

To achieve a completely integrated, standalone package, certainimplementations of the system 10 affix 300 μm×300 μm×100 μm thin LEDS(Rothiner LaserTechnik, CHIP-980-P50) to the chip backside—illuminatingdirectly through the chip, eliminating the need for waveguides or fiberoptics. At 980 nm, these commercially available components produce 75W/cm² [68 mW/(300 μm)²]. These LEDs can be used, but require averagingof multiple images (for example 6 images with an LED of power 75 W/cm²is equivalent to a single image of 400 W/cm²). Similarly, 1550 nm LEDs(Seminex, CHP-124) produce 22 W/cm² (pulsed versions reach up to 200W/cm², CHP-157). Aided by angle selective gratings, the new highersensitivity imager achieves a total of 20× increase in spatialresolution to 12 μm for single cell resolution.

Thinning the prototype imager to 25 μm, the various implementations ofthe system 10 have demonstrated through illumination at 1550 and 980 nm.Increased sensitivity allows reduction (10X) in illumination power. Thevarious implementations of the system 10 anticipate detection of asingle cell with only 25×10³ immunotargeted UCNPs for ultrasensitivedetection—expanding use to a wide array of cell types.

No appreciable interaction of light with silicon is observed at 1550 nm,as the characteristic absorption depth is ˜100 m, however somebackground is generated at 980 nm (absorption depth 100 um).Contribution of 980 nm induced background through use of (1) a new diodestructure (P-N-P) that shields the detector from substrate carriers, and(2) a transient current “sink” that drains any substrate charge beforeimaging. Microfabricated mirrors can be added to redirect LED outputlight. These features are discussed above Given the sub-micron featuresof modern CMOS processes, reduction of pixel size to 20 μm is readilyachieved. Fill-factor is reduced (due to the in-pixel amplifier) from85% currently, to 70%. However, this translates into an interpixeldistance of <5 μm, and therefore all cells will be at least partiallycaptured by a pixel. Power consumption (currently just 3.5 mW) willincrease by 4×, but well within acceptable limits (14 mW).

Integrated Surgical Tools for Real-Time Cell Imaging with ImmunotargetedUCNPs. The various implementations of the system 10 require an imagerthat can be used during surgery that is sufficiently small and iscompatible with existing methods, and require minimal disruption toclinical practice. Several instantiations of the device can beenvisioned. For example, one approach is to embed the imager along abiopsy needle and scalpel, as both of these instruments are used insurgery and thus can be easily integrated. This will be done by customfabricating the imager in a form-factor that covers the surface of ascalpel blade; the various implementations of the system 10 then affixan imager to each side, with only a set of thin wires running along thehandle for power supply and data transfer. Made in batch, CMOStechnology is extremely low cost, allowing these devices to besingle-use, disposable items. Other implementations include directintegration on a probe surface, thinning the device enough to beflexible and fitting on a curved surface, mounting on multiple sides ofa probe, attaching to a robotic instrument, and directly integratinginto surgical gloves.

3D Positional Sensing. Various implementations incorporate a method ofrecording the real-time position of the imager so that the threedimensional spatial coordinates of each image gathered is recorded.These images are then assembled in real-time to create athree-dimensional construct of the tissue or surface imaged.

The position sensor can be incorporated using CMOS compatibletechniques, such as inclusion of an accelerometer and gyroscope on chip,or packaged separately and attached to the imaging chip. Additionally,or alternatively, the probe (or device/platform that the chip is mountedto) can record positional information, through either attachment of aposition sensor (combination of accelerometer and gyroscope) ormarkers/fiducials attached to the device that allow registration in 3Dspace.

Ranges can be expressed herein as from “about” one particular value,and/or to “about” another particular value. When such a range isexpressed, a further aspect includes from the one particular valueand/or to the other particular value. Similarly, when values areexpressed as approximations, by use of the antecedent “about,” it willbe understood that the particular value forms a further aspect. It willbe further understood that the endpoints of each of the ranges aresignificant both in relation to the other endpoint, and independently ofthe other endpoint. It is also understood that there are a number ofvalues disclosed herein, and that each value is also herein disclosed as“about” that particular value in addition to the value itself. Forexample, if the value “10” is disclosed, then “about 10” is alsodisclosed. It is also understood that each unit between two particularunits are also disclosed. For example, if 10 and 15 are disclosed, then11, 12, 13, and 14 are also disclosed.

As used herein, the term “subject” refers to the target ofadministration, e.g., an animal. Thus, the subject of the hereindisclosed methods can be a human, non-human primate, horse, pig, rabbit,dog, sheep, goat, cow, cat, guinea pig or rodent. The term does notdenote a particular age or sex. Thus, adult and newborn subjects, aswell as fetuses, whether male or female, are intended to be covered. Inone aspect, the subject is a mammal. A patient refers to a subjectafflicted with a disease or disorder. The term “patient” includes humanand veterinary subjects.

Although the disclosure has been described with reference to preferredembodiments, persons skilled in the art will recognize that changes maybe made in form and detail without departing from the spirit and scopeof the disclosed apparatus, systems and methods.

TABLE 1 SNR. Illumination Wavelength Current Anticipated ΔSNR (dB) 9801550 980 1550 980 1550 Image time  4 ms  4 ms Spatial Resolution 250 μm  12 μm Cell to sensor   2 mm <0.05 mm   +31 distance *Power (W/cm², 2ms) 6 65 420 500 +19 +9 Optical Sensitivity 0.05 A/W 0.52 A/W +10 CellsImaged 1 × 10⁷ 1 −70 Noise Reduction 6 Baseline SNR 26 dB 23 dB SNR forSingle Cell 22 9 (10⁶ UCNPs/cell) Min UCNPs de-tectable 25K 500K (SNR =6 dB) Improvements in SNR and minimum number of UCNPs detectable withthe imager. Image time: 2 ms illuminati ms readout. Single cell imagingwith just 25,000 UCNPs/cell is possible at 980. *American NationalStandards Institute limits for a 2 millisecon pulse.

What is claimed is:
 1. An imaging system, comprising: a. an imagingchip; and b. a composition comprising at least one optical nanoparticlewherein the imaging chip is constructed and arranged to detect lightemission from that nanoparticle.
 2. The imaging system of claim 1,wherein the imaging chip is configured for time-resolved imaging.
 3. Theimaging system of claim 2, wherein the imaging system further comprisesan illumination device.
 4. The imaging system of claim 1, wherein theimaging chip is filterless.
 5. The imaging system of claim 1, whereinthe at least one optical nanoparticle has a luminescence lifetimegreater than 1 microsecond.
 6. The imaging system of claim 1, whereinthe at least one optical nanoparticle has a luminescence lifetimegreater than 10 microseconds.
 7. The imaging system of claim 1, whereinthe at least one optical nanoparticle comprises an upconvertingnanoparticle (UCNP).
 8. The imaging system of claim 1, wherein the atleast one optical nanoparticle upconverts near-infrared light to higherenergy light.
 9. The imaging system of claim 1, wherein the imaging chipis lensless.
 10. The imaging system of claim 1, wherein the imaging chipis between about 25 microns and about 1 mm thick.
 11. The imaging systemof claim 1, wherein the imaging chip is between about 1 mm{circumflexover ( )}2 and about 40 cm{circumflex over ( )}2 wide.
 12. The imagingsystem of claim 1, wherein the imaging chip is fitted to a medicaldevice.
 13. The imaging system of claim 12, wherein the medical deviceis a scalpel, probe, drop in probe for laparoscopic or robotic surgery,or glove.
 14. The imaging system of claim 1, further comprising anillumination array.
 15. The imaging system of claim 14, wherein theillumination array is an array of LED or laser diodes surrounding thechip.
 16. The imaging system of claim 14, wherein the illumination arrayis an array of LEDs, laser diodes, or fiber optics configured forthrough illumination.
 17. The imaging system of claim 14, wherein thethrough illumination is patterned.
 18. The imaging system of claim 1,wherein the optical nanoparticle is conjugated to a molecule targetedtoward a cell type of interest.
 19. The imaging system of claim 18,wherein the molecule is a protein, antibody, component of an antibody,or small molecule.
 20. The imaging system of claim 19, wherein the celltype of interest is a cancer cell.
 21. A method of imaging diseasetissue, comprising: a) introducing a composition comprising at least oneoptical nanoparticle into tissue; b) illuminating the at least oneoptical nanoparticle; and c) recording luminescence of the at least oneoptical nanoparticle with an imaging chip.
 22. The method of claim 21,wherein the at least one optical is illuminated at a first wavelengthand emits light at a second wavelength.
 23. The method of claim 22,wherein the illumination wavelength is longer than the emittedwavelength.
 24. The method of claim 21, wherein the luminescence isrecoded after the at least one optical nanoparticle is illuminated. 25.The method of claim 21, wherein at least one optical nanoparticle is anupconverting nanoparticle.
 26. The method of claim 21, furthercomprising performing ratiometric imaging to determine the depth of theat least one optical nanoparticle.
 27. The method of claim 21, whereinat least one optical nanoparticle is conjugated to a molecule or proteinthat binds to a target cell.
 28. The method of claim 21, wherein theprotein is an antibody or derivative.